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Abstract 

An  attack  graph  is  a  succinct  representation  of  all  paths  through  a  system  that  end  in  a  state  where  an  intruder 
has  successfully  achieved  his  goal.  Today  Red  Teams  determine  the  vulnerability  of  networked  systems 
by  drawing  gigantic  attack  graphs  by  hand.  Constructing  attack  graphs  by  hand  is  tedious,  error-prone, 
and  impractical  for  large  systems.  By  viewing  an  attack  as  a  violation  of  a  safety  property,  we  can  use 
model  checking  to  produce  attack  graphs  automatically:  a  successful  path  from  the  intruder’s  viewpoint  is  a 
counterexample  produced  by  the  model  checker.  In  this  paper  we  present  an  algorithm  for  generating  attack 
graphs  using  model  checking. 

Security  analysts  use  attack  graphs  for  detection,  defense,  and  forensics.  In  this  paper  we  present  a  minimiza¬ 
tion  technique  that  allows  analysts  to  decide  which  minimal  set  of  security  measures  would  guarantee  the 
safety  of  the  system.  We  provide  a  formal  characterization  of  this  problem:  we  prove  that  it  is  polynomially 
equivalent  to  the  minimum  hitting  set  problem  and  we  present  a  greedy  algorithm  with  provable  bounds.  We 
also  present  a  reliability  technique  that  allows  analysts  to  perform  a  simple  cost-benefit  analysis  depending 
on  the  likelihoods  of  attacks.  By  interpreting  attack  graphs  as  Markov  Decision  Processes  we  can  use  a  stan¬ 
dard  MDP  value  iteration  algorithm  to  compute  the  probabilities  of  intruder  success  for  each  attack  the  graph. 

We  illustrate  our  work  in  the  context  of  a  small  example  that  includes  models  of  a  firewall  and  an  intrusion 
detection  system. 


important  to  automate.  When  evaluating  the  security  of  a  network,  it  is  not  enough  to  consider  the  presence 
or  absence  of  isolated  vulnerabilities.  A  large  network  builds  upon  multiple  platforms  and  diverse  software 
packages  and  supports  several  modes  of  connectivity.  Inevitably,  such  a  network  will  contain  security  holes 
that  have  escaped  notice  of  even  the  most  diligent  system  administrator. 


Attack  Graph 


Figure  1 :  Vulnerability  Analysis  of  a  Network 

To  evaluate  the  vulnerability  of  a  network  of  hosts,  a  security  analyst  must  take  into  account  the  effects  of 
interactions  of  local  vulnerabilities  and  find  global  vulnerabilities  introduced  by  interconnections.  A  typical 
process  for  vulnerability  analysis  of  a  network  is  shown  in  Figure  1 .  First,  scanning  tools  determine  vulnera¬ 
bilities  of  individual  hosts.  Using  this  local  vulnerability  information  along  with  other  information  about  the 
network,  such  as  connectivity  between  hosts,  the  analyst  produces  an  attack  graph.  Each  path  in  an  attack 
graph  is  a  series  of  exploits,  which  we  call  atomic  attacks,  that  leads  to  an  undesirable  state  (e.g.,  a  state 
where  an  intruder  has  obtained  administrative  access  to  a  critical  host). 

1.1  Attack  Graphs  and  Intrusion  Detection 

Attack  graphs  can  serve  as  a  basis  for  detection,  defense,  and  forensic  analysis.  To  motivate  our  study  of  the 
generation  and  analysis  of  attack  graphs,  we  discuss  the  potential  applications  of  attack  graphs  to  these  areas 
of  security. 

Detection 

System  administrators  are  increasingly  deploying  intrusion  detections  systems  (IDSs)  to  detect  and  combat 
attacks  on  their  network.  Such  systems  depend  on  software  sensor  modules  that  first  detect  suspicious  events 
and  activity  and  then  issue  alerts.  Setting  up  the  sensors  involves  a  trade-off  between  sensitivity  to  intrusions 
and  the  rate  of  false  alarms  in  the  alert  stream.  When  the  sensors  are  set  to  report  all  suspicious  events,  the 
sensors  frequently  issue  alerts  for  benign  background  events.  Frequent  false  alarms  results  in  administrators 
turning  off  the  IDS  entirely.  On  the  other  hand,  decreasing  sensor  sensitivity  reduces  their  ability  to  detect 
real  attacks. 

To  address  this  trade-off,  many  intrusion  detection  systems  employ  heuristic  algorithms  to  correlate  alerts 
from  a  large  pool  of  heterogeneous  sensors.  Valdes  and  Skinner  [VS01]  describe  a  probabilistic  approach 
to  alert  correlation.  Successful  correlation  of  multiple  alerts  increases  the  chance  that  the  suspicious  activity 
indicated  by  the  alerts  is  in  fact  malicious. 

Attack  graphs  can  enhance  both  heuristic  and  probabilistic  correlation  approaches.  Given  a  graph  de¬ 
scribing  all  likely  attacks  (i.e.,  sequences  of  attacker  actions),  an  IDS  can  match  individual  alerts  to  attack 
edges  in  the  graph.  Matching  successive  alerts  to  individual  paths  in  the  attack  graphs  dramatically  increases 
the  likelihood  that  the  network  is  under  attack.  This  on-line  vigilance  allows  the  IDS  to  predict  attacker  goals, 
aggregate  alarms  to  reduce  the  volume  of  alert  information  to  be  analyzed,  and  reduce  the  false  alarm  rates. 
Knowledge  of  attacker  goals  and  likely  next  steps  helps  guide  defensive  response. 


our  models  are  expressive  enough  to  reflect  the  administrator’s  choice  of  security  policy  for  an  IDS  and  his 
choice  of  network  configuration.  Attack  graphs  enable  an  administrator  to  perform  several  kinds  of  analyses 
to  assess  their  security  needs:  marking  the  paths  in  the  attack  graph  that  an  IDS  will  detect;  determining 
where  to  position  new  IDS  components  for  best  coverage;  exploring  trade-offs  between  different  security 
policies  and  between  different  software/hardware  configurations;  and  identifying  the  worst-case  scenarios 
and  prioritizing  defense  strategry  accordingly. 

Forensics 

After  a  break-in,  forensic  analysis  is  used  to  find  probable  attacker  actions  and  to  assess  damage.  If  legal  ac¬ 
tion  is  desired,  analysts  seek  evidence  that  a  sequence  of  sensor  alerts  comprises  a  coherent  attack  plan,  and 
is  not  merely  a  series  of  isolated,  benign  events.  This  task  becomes  even  harder  when  the  intruders  obfuscate 
attack  steps  by  slowing  down  the  pace  of  the  attack  and  varying  specific  steps.  We  can  construct  a  convincing 
argument  as  to  the  malicious  intent  of  intruder  actions  by  matching  data  extracted  from  IDS  logs  to  a  formal 
reference  model  based  on  attack  graphs  [Ste]. 

Given  that  attack  graphs  can  be  used  to  perform  a  variety  of  analysis,  we  can  use  them  to  answer  the 
following  kinds  of  questions,  of  particular  interest  to  system  administrators: 

Question  1:  What  successful  attacks  are  undetected  by  the  IDS? 

Question  2:  If  all  measures  for  protecting  a  network  are  deployed,  does  the  system  become  safe? 

Question  3:  Given  a  set  of  measures  M,  what  is  the  smallest  subset  of  measures  M'  whose  deployment 
makes  the  system  safe? 

Answers  to  these  questions,  can  help  a  system  or  network  administrator  choose  the  best  upgrade  strategy. 
We  address  these  questions  in  Section  5. 

When  we  are  modeling  a  system  operating  in  an  unpredictable  environment,  certain  transitions  in  the 
model  represent  the  system’s  reaction  to  changes  in  the  environment.  We  can  think  of  such  transitions  as 
being  outside  of  the  system’s  control — they  occur  when  triggered  by  the  environment.  When  no  empirical 
information  is  available  about  the  relative  likelihood  of  such  environment-driven  transitions,  we  can  model 
them  only  as  nondeterministic  “choices”  made  by  the  environment.  Moreover,  for  new  vulnerabilities  data 
for  estimating  likelihoods  might  not  be  available.  However,  sometimes  empirical  data  make  it  possible  to 
assign  probabilities  to  environment-driven  transitions.  We  would  like  to  take  advantage  of  such  quantitative 
information  added  appropriately  to  attack  graphs.  In  this  context,  a  system  administrator  might  be  interested 
in  answering  the  following  question: 

Question  4:  The  deployment  of  which  security  measure(s)  will  increase  the  likelihood  of  thwarting  an  at¬ 
tacker? 

The  system  administrator  can  use  the  answer  to  question  4  to  perform  a  quantitative  evaluation  of  various 
security  fixes.  We  address  this  question  in  Section  6.2. 

1.2  Our  Contributions 

Constructing  attack  graphs  is  a  crucial  part  of  performing  vulnerability  analysis  of  a  network  of  hosts.  Cur¬ 
rently,  Red  Teams  produce  attack  graphs  by  hand,  often  drawing  gigantic  diagrams  on  floor-to-ceiling  white¬ 
boards.  Doing  this  by  hand  is  tedious,  error-prone,  and  impractical  for  attack  graphs  larger  than  a  hundred 
nodes. 

The  main  contributions  of  our  work,  some  of  which  have  appeared  in  an  earlier  paper  [SH.I+02]  are: 

•  We  demonstrate  how  model  checking  can  be  applied  to  generate  attack  graphs  automatically.  We  show 
that  the  attack  graphs  produced  by  our  method  are  exhaustive,  i.e.,  covering  all  possible  attacks,  and 
succinct,  i.e.,  containing  only  relevant  states  and  transitions  (see  Section  3.2). 


model  represents  the  state  of  the  system  between  atomic  attacks.  A  typical  transition  from  state  si 
to  state  so  corresponds  to  an  atomic  attack  whose  preconditions  are  satisfied  in  §i  and  whose  effects 
hold  in  state  sn.  An  attack  is  a  sequence  of  state  transitions  culminating  in  the  intruder  achieving  his 
goal.  The  entire  attack  graph  is  thus  a  representation  of  all  the  possible  ways  in  which  the  intruder  can 
succeed. 

•  We  prove  that  finding  a  minimum  set  of  atomic  attacks  that  must  be  removed  to  thwart  an  intruder  is 
.Y  /’-complete.  Beyond  the  proof  sketched  in  our  earlier  paper  [SHJ+02],  here  we  further  explore  the 
complexity  of  this  problem.  Section  5.2.1  proves  that  the  problem  is  polynomially  equivalent  to  the 
minimum  hitting  set  problem  where  the  collection  of  sets  is  represented  as  a  labeled  directed  graph. 
This  reduction  provided  us  with  additional  insight,  enabling  us  to  find  a  greedy  algorithm  with  provable 
bounds,  which  can  be  used  to  answer  questions  1,  2,  and  3. 

•  We  present  an  algorithm  to  compute  the  reliability — fined  as  the  likelihood  of  an  intruder  not  succeeding- 
of  a  networked  system.  An  advantage  of  our  algorithm  is  that  it  allows  incomplete  information,  i.e., 
probabilities  of  all  transitions  need  not  be  provided.  To  our  knowledge,  previous  metrics  in  the  area  of 
security  require  complete  information.  We  can  use  this  algorithm  an  answer  question  4  precisely. 

We  present  related  work  in  Section  2.  Section  3  describes  our  model  and  our  algorithm  to  generate  attack 
graphs.  We  give  details  of  an  example  networked  system  in  Section  4  and  use  it  throughout  the  paper  for 
illustrative  purposes.  In  Section  5  we  present  a  minimization  analysis  to  help  administrators  decide  what 
measures  to  deploy  to  thwart  attacks.  In  Section  6  we  present  a  reliability  analysis  over  probabilistic  attack 
graphs  based  on  the  value  iteration  algorithm  defined  for  Markov  Decision  Processes;  this  analysis  can  help 
administrators  determine  how  deployment  of  one  measure  can  decrease  the  likelihood  of  certain  attacks. 
Finally,  we  present  a  brief  summary  and  directions  for  future  work  in  Section  7. 


2  Related  Work 

Phillips  and  Swiler  [PS98]  propose  a  concept  of  attack  graphs  similar  to  the  one  we  describe.  However,  they 
model  only  attacks.  Since  we  have  a  generic  state  machine  model,  we  can  simultaneously  model  not  just 
attacks,  but  also  seemingly  benign  system  events  (e.g.,  link  failures  and  user  errors)  and  even  system  admin¬ 
istrator  recovery  actions.  Therefore,  our  attack  graphs  are  more  general  than  the  one  proposed  by  Phillips  and 
Swiler.  They  also  built  a  tool  for  generating  attack  graphs  [SPECOO];  it  constructs  the  attack  graph  by  for¬ 
ward  exploration  starting  from  the  initial  state.  In  our  work,  we  use  a  symbolic  model  checker  (i.e.,  NuSMV) 
that  works  backward  from  the  goal  state  to  construct  the  attack  graph.  A  major  advantage  of  the  backward 
algorithm  is  that  vulnerabilities  that  are  not  relevant  to  the  safety  property  (or  the  goal  of  the  intruder)  are 
never  explored;  this  technique  can  result  in  significant  savings  in  space.  In  fact,  Swiler  el  al.  [SPECOO]  refer 
to  the  advantages  of  the  backward  search  in  their  paper.  Finally,  the  post-facto  analysis  suggested  by  Phillips 
and  Swiler  is  also  different  from  the  ones  we  present  in  this  paper.  We  plan  to  incorporate  their  analysis  into 
our  tool  suite. 

Dacier  [Dac94]  proposes  the  concept  of  privilege  graphs,  where  each  node  represents  a  set  of  privileges 
owned  by  the  user  and  arcs  represent  vulnerabilities.  Privilege  graphs  are  then  explored  to  construct  attack 
state  graphs,  which  represent  different  ways  in  which  an  intruder  can  reach  a  certain  goal,  such  as  root  access 
on  a  host.  Dacier  proposes  a  metric,  called  the  mean  effort  to  failure  or  METF,  based  on  the  attack  state 
graphs.  Orlato  el  al.  [ODK99]  describe  an  experimental  evaluation  of  this  framework.  At  the  surface  our 
notion  of  attack  graphs  seems  similar  to  Dacier’s.  However,  as  in  the  case  with  Phillips  and  Swiler,  Dacier 
takes  an  “attack-centric”  view  of  the  world;  again,  our  attack  graphs  are  more  general.  From  the  experiments 
conducted  by  Orlato  el  al.  it  appears  that  even  for  small  examples  the  space  required  to  construct  attack 
state  graphs  becomes  prohibitive.  Model  checking  has  made  significant  advances  in  representing  large  state 
spaces.  Therefore,  by  basing  our  algorithm  on  model  checking  we  leverage  off  those  advances  and  can 
hope  to  represent  large  attack  graphs.  The  analytical  analysis  proposed  by  Dacier  can  also  be  performed  on 


used  the  unmodified  model  checker  SMV  [SMV].  Therefore,  they  could  only  obtain  one  counter-example 
or  one  attack  corresponding  to  a  intruder’s  goal.  In  contrast,  we  modified  the  model  checker  NuSMV  to 
produce  complete  attack  graphs,  which  represents  all  possible  attacks.  We  also  described  analyses  that  can 
be  performed  on  these  attack  graphs.  These  analyses  cannot  be  meaningfully  performed  on  single  attacks. 


3  Generating  Attack  Graphs  using  Model  Checking 

First,  we  formally  define  attack  graphs ,  the  data  structure  used  to  represent  all  possible  attacks  on  our  net¬ 
worked  system.  We  restrict  our  attention  to  attack  graphs  representing  violations  of  safety  properties1. 

Definition  1  Let  AP  be  a  set  of  atomic  propositions.  An  attack  graph  or  AG  is  a  tuple  G  =  (S,t,  So,  Ss,  L), 
where  S'  is  a  set  of  states,  t  C  S  x  S  is  a  transition  relation.  So  C  S  is  a  set  of  initial  states,  Ss  C  S  is  a  set 
of  success  states,  and  L  :  S  — >■  2AP  is  a  labeling  of  states  with  a  set  of  propositions  true  in  that  state. 

Unless  stated  otherwise,  we  assume  that  the  transition  relation  t  is  total.  We  define  an  execution  fragment 
as  a  finite  sequence  of  states  sosi  ••••*«  such  that  (s{,  s8'+i)  £  t  for  all  0  <  i  <  n.  An  execution  fragment 
with  so  £  So  is  an  execution,  and  an  execution  whose  final  state  is  in  Ss  is  an  attack,  i.e.,  the  execution 
corresponds  to  a  sequence  of  atomic  attacks  leading  to  the  intruder’s  goal  state.  Intuitively,  Ss  denotes  all 
states  where  the  intruder  has  achieved  his  goal,  e.g.,  obtaining  root  access  on  a  critical  host. 

Next  we  turn  our  attention  to  algorithms  for  automatic  generation  of  attack  graphs  and  properties  that 
we  can  guarantee  of  them.  Starting  with  a  description  of  a  network  model  M  and  a  security  property  p,  the 
task  is  to  construct  an  attack  graph  representing  all  executions  of  M  that  violate  p — these  are  the  successful 
attacks.  For  the  kinds  of  attack  graph  analyses  suggested  in  Section  1,  it  is  essential  that  the  graphs  produced 
by  the  algorithms  be  exhaustive  and  succinct.  An  attack  graph  is  exhaustive  with  respect  to  a  model  M  and 
correctness  property  p  if  it  covers  all  possible  attacks  in  M  leading  to  a  violation  of  p,  and  succinct  if  it  only 
contains  those  states  and  transitions  of  M  that  lead  to  a  state  violating  p. 


3.1  Reachability  Analysis 

If  we  restrict  ourselves  to  safety  properties,  an  attack  graph  may  be  constructed  by  performing  a  simple  state- 
space  search.  Starting  with  the  initial  states  of  the  model  M,  we  use  a  graph  traversal  procedure  (e.g.,  depth 
first  search)  to  find  all  reachable  success  states  where  the  safety  property  p  is  violated.  The  attack  graph  is 
the  union  of  all  paths  from  initial  states  to  success  states. 

While  this  algorithm  has  the  advantage  of  simplicity,  it  handles  only  safety  properties  and  may  run  into 
the  state  explosion  problem  for  non-trivial  models.  Model  checking  has  dealt  with  both  of  these  issues  with 
some  success,  so  we  will  consider  algorithms  based  on  that  technology. 


3.2  Model  Checking  Algorithm 

Model  checking  is  a  technique  for  checking  whether  a  formal  model  M  of  a  system  satisfies  a  given  property 
p.  In  our  work,  we  use  the  model  checker  NuSMV  [NuS],  for  which  the  model  M  is  a  finite  labeled  transition 
system  and  p  is  a  property  expressed  in  Computation  Tree  Logic  ( CTL ).  For  now,  we  consider  only  safety 
properties,  which  in  CTL  have  the  form  AG/  (i.e.,  p  =  AG/,  where  /  is  a  formula  in  propositional  logic). 
If  the  model  M  satisfies  the  property  p,  NuSMV  reports  “true.”  If  M  does  not  satisfy  p,  NuSMV  produces  a 
counter-example.  A  single  counter-example  shows  an  execution  that  leads  to  a  violation  of  the  property.  In 
this  section,  we  explain  how  to  construct  attack  graphs  for  safety  properties  using  model  checking. 

1  We  say  more  on  liveness  properties  in  Section  7. 


So  C  S  -  set  of  initial  states 

L  :  S  — >■  2ap  -  labeling  of  states  with  propositional  formulas 
p  =  AG  (-^unsafe)  (a  safety  property) 

Output: 

attack  graph  GP  =  (S  unsafe ,  RP ,  ,Spf,L) 

Algorithm:  Generate AttackGraph(S ,  R ,  So,  L,p) 

(*  Use  model  checking  to  find  the  set  of  states  Sunsafe  that 
violate  the  safety  property  AG  (-^unsafe).  *) 

Sunsafe  =  nioclelGheck(S ,  R,  S0 ,  L ,  p) . 

(*  Restrict  the  transition  relation  R  to  states  in  the  set  Sunsafe  *) 
RP  =  RO  (Sunsafe  X  S  uns  a.fe  )  ■ 

Sq  —  So  Pi  Sunsafe • 

Spf  =  {s|s  E  Sunsafe  A  s  \=  unsafe}. 

return(S  unsafe,  Rp ,  Sp,  Sp ,  L). 


Figure  2:  Algorithm  for  Generating  Attack  Graphs 


Attack  graphs  depict  ways  in  which  the  system  can  reach  an  unsafe  state  (or,  equivalently,  a  successful 
state  for  the  intruder).  We  can  express  the  property  that  an  unsafe  state  cannot  be  reached  as: 

AG  (-^unsafe) 

When  this  property  is  false,  there  are  unsafe  states  that  are  reachable  from  the  initial  state.  The  precise  mean¬ 
ing  of  unsafe  depends  on  the  application.  For  example,  in  the  network  security  example  given  in  Section  4, 
the  property  given  below  is  used  to  express  that  the  privilege  level  of  the  intruder  on  the  host  with  index  2 
should  always  be  less  than  the  root  (administrative)  privilege. 

AG (network.adversary.privilege[2 ]  <  network.priv.root ) 

We  briefly  describe  the  algorithm  (see  Figure  2)  for  constructing  attack  graphs  for  the  property  AG (-*  unsafe ). 
The  first  step  is  to  determine  the  set  of  states  Sr  that  are  reachable  from  the  initial  state.  Next,  the  algorithm 
computes  the  set  of  reachable  states  Sunsafe  that  have  a  path  to  an  unsafe  state.  The  set  of  states  Sunsafe  is 
computed  using  an  iterative  algorithm  derived  from  a  fix-point  characterization  of  the  AG  operator  [CGPOO], 
Let  R  be  the  transition  relation  of  the  model,  i.e.,  (s,  s')  E  R  if  and  only  if  there  is  a  transition  from  state  s 
to  s'.  By  restricting  the  domain  and  range  of  R  to  S unsafe  we  obtain  a  transition  relation  RP  that  represents 
the  edges  of  the  attack  graph.  Therefore,  the  attack  graph  is  ( Su„safe ,  Rp ,  So\  Sp,  L),  where  Sunsafe  and  RP 
represent  the  set  of  nodes  and  edges  of  the  graph  respectively;  Sq  =  S o  fl  Sunsafe  is  the  set  of  initial  states; 
and  Sp  =  {s|s  E  Sunsafe  As  |=  unsafe}  is  the  set  of  success  states. 

In  symbolic  model  checkers,  such  as  NuSMV,  the  transition  relation  and  sets  of  states  are  represented 
using  BDDs  [Bry86],  a  compact  representation  for  boolean  functions.  There  are  efficient  BDD  algorithms 
for  all  operations  used  in  the  algorithm  shown  in  Figure  2. 


3.3  Attack  Graph  Properties 

We  can  show  that  an  attack  graph  G  generated  by  the  algorithm  in  Figure  2  is  exhaustive  (Lemma  1(a))  and 
succinct  with  respect  to  states  and  transitions  (Lemmas  1(b)  and  1(c)). 


an  attack  in  G  that  contains  s. 

(c)  succinct-transition.  A  transition  /  =  (si,s2)  of  the  input  model  (S,  R,  So,  L)  is  in  the  attack  graph  G  if 
and  only  if  there  is  an  attack  in  G  that  includes  /. 

Proof: 

(a)  exhaustive.  (=>)  Let  e  =  solo  •  •  •  t„.-isn  be  a  (finite)  execution  of  the  input  model  such  that  sn  is  an 

unsafe  state.  To  prove  that  e  is  an  attack  in  G,  it  is  sufficient  to  show  (1)  so  £  Sq,  (2)  sn  £  ,  and  (3)  for  all 

0  <  k  <  n,  Sk  £  S  and  tk  £  RP . 

Since  unsafe  holds  at  sn  and  for  all  k  there  is  a  path  from  sk  to  sn  in  the  input  model,  by  definition  every 
Sk  along  e  1  violates  AG(-> unsafe).  Therefore,  by  construction,  every  sk  is  in  Sunsafe  and  every  tk  is  in  RP . 
(1)  and  (2),  and  (3)  follow  immediately. 

(-4=)  Suppose  that  e  =  .s . . .  /  „ _  i  sn  is  an  attack  in  the  attack  graph  G.  By  construction,  all  states  and 
transitions  of  e  are  also  states  and  transitions  in  the  input  model.  Since  e  is  an  attack,  so  £  So  and  sn  £  . 

Therefore,  so  £  So  and  sn  £  S.  So  e  is  an  execution  of  the  input  model,  its  first  state  is  an  initial  state  of  the 
model,  and  p  is  false  in  its  final  state.  It  follows  that  e  violates  the  property  AG  (-in  ns  afe). 

(b)  succinct-state.  (=>)  By  construction  of  the  algorithm  in  Figure  2,  all  states  generated  for  the  attack 
graph  are  reachable  from  an  initial  state,  and  all  of  them  violate  AG  (-^unsafe).  Therefore,  for  any  such  state 
s  in  the  input  model,  there  is  a  path  e\  from  an  initial  state  to  s,  and  there  is  a  path  e2  from  s  to  an  unsafe 
state. 

The  concatenation  of  e\  and  e2  is  an  execution  e  of  the  input  model  that  violates  AG  (-^unsafe).  By 
lemma  la,  e  is  an  attack  in  G.  Since  e  contains  s,  the  proof  is  complete. 

( ■'=  )  If  there  is  an  attack  in  G  that  contains  s,  then  trivially  s  is  in  G. 

(c)  succinct-transition.  (=>)  By  lemma  lb,  there  is  an  attack  t  \  =  7 . . .  si  . . .  1  qm  that  contains 

state  si  and  an  attack  e2  =  rouo  . . .  s2  . . .  wn_i?’n  that  contains  state  s2.  So  the  following  attack  includes 
both  states  si  and  s2  and  the  transition  t:  e  =  qoto  ■  ■  ■  s\tsy , . .  wn_i?’n. 

( ■'=)  Tf  there  is  an  attack  in  G  that  contains  /,  then  trivially  /  is  in  G. 

□ 


4  A  Simple  Intrusion  Detection  Example 

Consider  the  example  network  shown  in  Figure  3.  There  are  two  target  hosts,  ip  and  *p2,  and  a  firewall 
separating  them  from  the  rest  of  the  Internet.  As  shown,  each  host  is  running  two  of  three  possible  services 
(ftp,  sshd,  a  database).  An  intrusion  detection  system  (IDS)  monitors  the  network  traffic  between  the  target 
hosts  and  the  outside  world.  There  are  four  possible  atomic  attacks,  identified  numerically  as  follows:  (0) 
sshd  buffer  overflow,  (1)  ftp  .rhosts,  (2)  remote  login,  and  (3)  local  buffer  overflow.  If  an  atomic  attack  is 
detectable,  the  intrusion  detection  system  will  trigger  an  alarm;  if  an  attack  is  stealthy ,  the  IDS  misses  it. 
The  ftp  .rhosts  attack  needs  to  find  the  target  host  with  two  vulnerabilities:  a  writable  home  directory  and 
an  executable  command  shell  assigned  to  the  ftp  user  name.  The  local  buffer  overflow  exploits  a  vulnerable 
version  of  the  xterm  executable. 

In  this  section,  we  construct  a  finite  state  model  of  the  example  network  so  that  each  state  transition 
corresponds  to  a  single  atomic  attack  by  the  intruder.  A  state  in  the  model  represents  the  state  of  the  system 
between  atomic  attacks.  A  typical  transition  from  state  si  to  state  s2  corresponds  to  an  atomic  attack  whose 
preconditions  are  satisfied  in  si  and  whose  effects  hold  in  state  s2. 

The  intruder  launches  his  attack  starting  from  a  single  computer,  ipa,  which  lies  outside  the  firewall.  His 
eventual  goal  is  to  disrupt  the  functioning  of  the  database.  For  which,  the  intruder  needs  root  access  on  the 
database  host  *p2 . 


Figure  3:  Example  Network 


4.1  States  of  the  Finite  State  Machine  Model 

The  Network 

We  model  the  network  as  a  set  of  facts,  each  represented  as  a  relational  predicate.  The  state  of  the  network 
specifies  services,  host  vulnerabilities,  connectivity,  and  a  remote  login  trust  relationship  between  hosts. 
There  are  six  boolean  variables  for  each  host,  specifying  whether  any  of  the  three  modeled  services  are 
running  and  whether  any  vulnerabilities  are  present  on  that  host. 


variable 

meaning 

sshh 

ssh  service  is  running  on  host  h 

ftp* 

ftp  service  is  running  on  host  h 

data/j 

database  is  running  on  host  h 

wdir/j 

ftp  home  directory  is  writable  on  host  h 

fshell/j 

ftp  user  has  executable  shell  on  host  h 

xterm/j 

xterm  executable  is  vulnerable  to  overflow  on  host  h 

Connectivity  is  expressed  as  a  ternary  relation  R  C  Host  x  Host  x  Port,  where  R{hi,  hn,p)  means  that 
host  h  2  is  reachable  from  host  li  on  port  p.  The  constants  sp  and  fp  will  refer  to  the  specific  ports  for  the 
ssh  and  ftp  services,  respectively.  Slightly  abusing  notation  (by  overloading  R),  we  write  It  (h  \ .  hP)  when 
there  is  a  network  route  from  hi  to  /x. .  We  model  trust  as  a  binary  relation  RshTrust  C  Host  x  Host,  where 
RsliTrust(hi ,  h%)  indicates  that  a  user  may  log  in  from  host  h?  to  host  hi  without  authentication  (i.e.,  host  h 
“trusts’'  host  hn). 

The  Intruder 

The  function plvl,\ :  Hosts  — >■  {none,  user,  root}  gives  the  level  of  privilege  that  intruder  /l  has  on  each  host. 
There  is  a  total  order  on  the  privilege  levels:  none  <  user  <  root. 

Several  state  variables  specify  which  attack  the  intruder  will  attempt  next: 


variable 

meaning 

attack 

attack  type 

source 

source  host 

target 

target  host 

strain 

stealthy  /detectable  attack 

detectable,  it  will  trigger  an  alarm  when  executed  on  a  host  or  network  segment  monitored  by  the  IDS;  if  an 
attack  is  stealthy ,  the  IDS  does  not  detect  it. 

We  specify  the  IDS  with  a  function  ids:  Host  x  Host  x  Attack  — >■  { d ,  s ,  &},  where  ids(h\ ,  ft2,  a)  =  d  if 
attack  a  is  detectable  when  executed  with  source  host  h i  and  target  host  hn',  ids(h\ ,  /)2,  a)  =  s  if  attack  a  is 
stealthy  when  executed  with  source  host  h\  and  target  host  hn;  and  ids(h i,  /)2,  a)  =  d  if  attack  a  has  doth 
detectable  and  stealthy  strains,  and  success  in  detecting  the  attack  depends  on  which  strain  is  used.  When 
hi  and  hn  refer  to  the  same  host,  ids(h\ ,  hn,  a)  specifies  the  intrusion  detection  system  component  (if  any) 
located  on  that  host.  When  h i  and  hn  refer  to  different  hosts,  ids(h\ ,  hn,  a)  specifies  the  intrusion  detection 
system  component  (if  any)  monitoring  the  network  path  between  h  and  hn.  In  addition,  a  global  boolean 
variable  specifies  whether  the  IDS  alarm  has  been  triggered  by  any  previously  executed  atomic  attack. 

4.2  Initial  States 

Initially,  there  is  no  trust  between  any  of  the  hosts;  the  trust  relation  Tr  is  empty.  The  connectivity  relation  R 
is  shown  in  the  following  table.  An  entry  in  the  table  corresponds  to  a  pair  of  hosts  (hi,hn).  Each  entry  is  a 
triple  of  boolean  values.  The  first  value  is  ‘y’  if  h  and  hn  are  connected  by  a  physical  link,  the  second  value 
is  ‘y’  if  h i  can  connect  to  hn  on  the  ftp  port,  and  the  third  value  is  ‘y’  if  h  can  connect  to  hn  on  the  sshd  port. 


R 

iPa 

ipi 

ipn 

ip  a 

y,n,n 

y.y.y 

y.y.n 

ipi 

y,n,n 

y.y.y 

y.y.n 

ipn 

y,n,n 

y-y-y 

y.y.n 

We  use  the  connectivity  relation  to  reflect  the  firewall  rule  sets  as  well  as  the  existence  of  physical  links. 
For  the  table  above,  the  firewall  is  open  and  does  not  place  any  restrictions  on  the  flow  of  network  traffic. 
Initially,  the  intruder  has  root  privileges  on  his  own  machine  ipa  and  no  privileges  on  the  other  hosts. 

The  paths  between  (ipa ,  ipi )  and  between  (ipa ,  ipn )  are  monitored  by  the  single  network-based  IDS.  The 
path  between  (ip  ,  ipn )  is  not  monitored.  There  are  no  other  host-based  intrusion  detection  components.  The 
IDS  detects  the  remote  login  attack  and  the  detectable  strains  of  the  sshd  buffer  overflow  attack. 


4.3  Transitions 

Our  model  has  nondeterministic  state  transitions.  If  the  current  state  of  the  network  satisfies  the  precondi¬ 
tions  of  more  than  one  atomic  attack  rule,  the  intruder  nondeterministically  “chooses”  one  of  those  attacks. 
The  state  then  changes  according  to  the  effects  clause  of  the  chosen  attack  rule.  The  intruder  repeats  this 
process  until  his  goal  is  achieved. 

We  model  four  atomic  attacks.  Throughout  the  description,  S  is  used  to  designate  the  source  host  and  T 
the  target  host.  Recall  that  R(S,T,p)  denotes  that  host  T  is  reachable  from  host  S  on  port  p. 

Sshd  Buffer  Overflow 


This  remote-to-root  attack  immediately  gives  a  remote  user  a  root  shell  on  the  target  machine. 


intruder  preconditions 

pIvIa(S)  >  user 
plvlA  (T)  <  root 

network  preconditions 

sshy 

R(S,  T,sp) 

intruder  effects 

plvlA  ( T )  =  root 

network  effects 

-isshy 

end 


User-level  privileges  on  host  S 
No  root-level  privileges  on  host  T 

Host  T  is  running  sshd 

Host  T  is  reachable  from  S  on  port  sp 

Root-le\’el  privileges  on  host  T 

Host  T  is  not  running  sshd 


Ftp  .rhosts 

Using  an  ftp  vulnerability,  the  intruder  creates  an  .rhosts  file  in  the  ftp  home  directory,  creating  a  remote  login 
trust  relationship  between  his  machine  and  the  target  machine. 


attack  ftp-rhosts  is 

intruder  preconditions 
pIvIa(S)  >  user 
network  preconditions 
ftPT 

R(S,T,fp) 

wdir/ 

fshell/ 

3 A' .~iRshTrust(X ,  T) 

intruder  effects 

none 

network  effects 

VA' .RshTrust(X ,  T) 

end 


User-level  privileges  on  host  S 

Host  T  is  running  ftp 

Host  T  is  reachable  from  S  on  port  fp 

Ftp  directory  writable  on  host  T 

Ftp  user  has  been  assigned  a  valid  shell  on  hostT 

No  rsh  trust  for  some  host  X  and  T 


Rsh  trust  between  all  hosts  and  T 


Remote  Login 

Using  an  existing  remote  login  trust  relationship  between  two  machines,  the  intruder  logs  in  from  one  machine 
to  another,  getting  a  user  shell  without  supplying  a  password.  This  operation  is  usually  a  legitimate  action 
performed  by  regular  users,  but  from  the  intruder’s  viewpoint,  it  is  an  atomic  attack. 


attack  rsh-login  is 

intruder  preconditions 

pIvIa(S)  =  user 
plvlA  ( T )  =  none 

network  preconditions 

RshTrust(S ,  T) 

R(S,  T) 

intruder  effects 


User-level  privileges  on  host  S 
No  privileges  on  host  T 

Rsh  trust  between  S  and  T 
Host  T  is  reachable  from  S 


end 


Local  Buffer  Overflow 

If  the  intruder  has  acquired  a  user  shell  on  the  target  machine,  the  next  step  is  to  exploit  a  buffer  overflow 
vulnerability  on  a  setuid  root  file  to  gain  root  access. 


attack  local-setuid-buffer-overflow  is 

intruder  preconditions 

pIvIa(T)  =  user  User-level  privileges  on  hostT 

network  preconditions 

xtermy  There  is  a  vulnerable  xterm  executable 

intruder  effects 

pIvIa(T)  =  root  Root-level  privileges  on  host  T 

network  effects 

none 


It  is  easy  to  see  that  each  atomic  attack  strictly  increases  either  the  intruder’s  privilege  level  on  the  target 
host  or  remote  login  trust  between  hosts.  This  means  that  the  attack  graph  has  no  cycles. 

From  our  finite  model  we  can  now  automatically  construct  attack  graphs  that  demonstrate  how  the  intruder 
can  violate  various  security  properties.  Suppose  we  want  to  generate  all  attacks  that  demonstrate  how  the 
intruder  can  gain  root  privilege  on  host  ipn  and  remain  undetected  by  the  IDS.  The  following  CTL  formula 
expresses  the  safety  property  that  the  intruder  on  host  ip-j  always  has  privilege  level  below  root  or  is  detected'. 

AG(network.adversary.privilege[2]  <  network.priv.root  \  network  .detected.) 


Figure  4  shows  the  attack  graph  produced  by  our  tool  for  this  property.  Each  node  is  labeled  by  an  attack 
id  number,  which  corresponds  to  the  atomic  attack  to  be  attempted  next',  a  flag  S/D  indicates  whether  the 
attack  is  stealthy  or  detectable  by  the  intrusion  detection  system;  and  the  numbers  of  the  source  and  target 
hosts  (ip a  corresponds  to  host  number  0). 

Any  path  in  the  graph  from  the  root  node  to  a  leaf  node  shows  a  sequence  of  atomic  attacks  that  the 
intruder  can  employ  to  achieve  his  goal  while  remaining  undetected.  For  instance,  the  path  highlighted  by 
dashed-boxed  nodes  consists  of  the  following  sequence  of  four  atomic  attacks:  overflow  sshd  buffer  on  host 
1,  overwrite  .rhosts  file  on  host  2  to  establish  rsh  trust  between  hosts  1  and  2,  log  in  using  rsh  from  host  1  to 
host  2,  and  finally,  overflow  a  local  buffer  on  host  2  to  obtain  root  privileges. 

We  have  also  expanded  the  example  described  above  by  adding  two  additional  hosts,  four  additional 
atomic  attacks,  several  new  vulnerabilities,  and  flexible  firewall  configurations.  For  this  larger  example  the 
attack  graph  has  5948  nodes  and  68364  edges. 


5  Minimization  Analysis 

Once  we  have  an  attack  graph  generated  for  a  specific  network  with  respect  to  a  given  safety  property,  we 
can  utilize  it  for  further  analysis.  A  system  administrator  has  available  to  him  a  set  of  measures,  such  as 
deploying  additional  intrusion  detection  tools,  adding  firewalls,  upgrading  software,  deleting  user  accounts. 


Figure  5:  Attack  Graph  Analysis 


set  of  atomic  attacks  they  thwart.  It  helps  us  answer  questions  such  as  1,  2,  and  3  posed  in  Section  1.1.  Let 
us  look  at  each  question  in  turn  since  they  suggest  different  solution  approaches. 

5.1  Minimal  Subsets  of  Atomic  Attacks  to  Thwart 

Suppose  we  want  to  find  a  minimal  set.  A,  of  atomic  attacks  that  must  be  prevented  to  guarantee  the  adver¬ 
sary  cannot  achieve  his  goal.  A  system  analyst  can  use  this  information  in  deciding  to  choose  one  measure 
mi,  which  eliminates  this  minimal  set  of  attacks  over  another  measure,  m 2,  perhaps  cheaper  than  mi,  but 
ineffective  with  respect  to  A. 

A  naive  solution  is  as  follows: 

1 .  Make  only  a  subset  of  the  atomic  attacks  available  to  the  intruder. 

2.  Run  the  model  checking  algorithm  to  determine  if  the  adversary  can  succeed. 

3.  Do  Steps  1  and  2  for  all  possible  non-empty  subsets  of  atomic  attacks. 

Clearly  this  solution  is  exponential  in  the  number  of  atomic  attacks.  For  our  example,  however,  the 
number  is  small,  and  we  can  easily  determine  this  minimal  set.  As  a  by-product  of  determining  this  set,  we 
can  easily  answer  the  first  question  posed  in  Section  1 . 

Question  1:  What  successful  attacks  are  undetected  by  the  IDS? 

Answer:  To  answer  this  question,  we  modify  the  model  slightly.  For  simplicity,  we  nondeterministically 
decide  which  subset  to  consider  initially,  before  any  attack  begins;  once  the  choice  is  made,  the  subset  of 
available  atomic  attacks  remains  constant  during  any  given  attack.  We  ran  the  model  checker  on  the  modified 
model  with  the  invariant  property  that  says  the  intruder  never  obtains  root  privilege  on  host  ip2 : 

AG(network.adversary.privilege[2 ]  <  network.priv.root ) 

The  post-processor  marked  the  states  where  the  intruder  has  been  detected  by  the  IDS.  The  result  is  shown 
in  Figure  5.  The  white  rectangles  indicate  states  where  the  attacker  had  not  yet  been  detected  by  the  intrusion 
detection  system.  The  black  rectangles  are  states  where  the  intrusion  detection  system  has  sounded  an  alarm. 
Thus,  white  leaf  nodes  are  desirable  for  the  attacker  because  his  objective  is  achieved  without  detection. 
Black  leaf  nodes  are  less  desirable — the  attacker  achieves  his  objective,  but  the  alarm  goes  off. 

The  resolution  of  which  atomic  attacks  are  available  to  the  intruder  happens  in  the  circular  nodes  near  the 
root  of  the  graph.  The  first  transition  out  of  the  root  (initial)  state  picks  the  subset  of  attacks  that  the  intruder 
will  use.  Each  child  of  the  root  node  is  itself  the  root  of  a  disjoint  subgraph  where  the  subset  of  atomic 
attacks  chosen  for  that  child  is  used.  Note  that  the  number  of  such  subgraphs  descending  from  the  root  node 
corresponds  to  the  number  of  subsets  of  atomic  attacks  with  which  the  intruder  can  be  successful — the  model 
checker  determines  that  for  any  other  possible  subset,  there  is  no  possible  successful  sequence  of  atomic 
attacks. 

The  root  of  the  graph  in  Figure  5  has  two  subgraphs,  corresponding  to  the  two  subsets  of  atomic  attacks 
that  will  allow  the  intruder  to  succeed.  In  the  left  subgraph  the  sshd  buffer  overflow  attack  is  not  available 
to  the  intruder;  it  can  be  readily  seen  that  the  intruder  can  still  succeed,  but  cannot  do  so  while  remaining 
undetected  by  the  IDS.  In  the  right  subgraph,  all  attacks  are  available.  Thus,  the  entire  attack  graph  implies 
that  all  atomic  attacks  other  than  the  sshd  attack  are  indispensable:  the  intruder  cannot  succeed  without  them. 
That  is,  for  no  other  subset  of  atomic  attacks  can  the  intruder  succeed  in  achieving  his  goal.  The  analyst  can 
use  this  information  to  guide  decisions  on  which  network  defenses  can  be  profitably  upgraded. 

The  white  cluster  in  the  middle  of  the  figure  is  isomorphic  to  the  attack  graph  presented  in  Figure  4;  it 
shows  attacks  in  which  the  intruder  can  achieve  his  objective  without  detection  (i.e.,  all  paths  by  which  the 
intruder  reaches  a  white  leaf  in  the  graph). 


AG  (—iimsafe) 


Let  A  be  the  set  of  atomic  attacks,  and  G  =  (S,  E ,  sq,ss,  L)  be  the  attack  graph,  where  S  is  the  set  of  states, 
E  C  S  x  S  is  the  set  of  edges,  so  E  S  is  the  initial  state,  ss  £  S'  is  the  success  state  for  the  intruder,  and 
L  :  E  — ►  A  U  {e}  is  a  labeling  function  where  L(e)  =  a  if  an  edge  e  =  (s  — y  s')  corresponds  to  an  atomic 
attack  a,  otherwise  L(e)  =  e.  Edges  labeled  with  <  represent  system  transitions  that  do  not  correspond 
to  an  atomic  attack.  Moreover,  as  demonstrated  below  additional  e  edges  can  be  also  introduced  by  our 
construction.  Without  loss  of  generality  we  can  assume  that  there  is  a  single  initial  and  success  state.  For 
example,  consider  an  attack  graph  with  multiple  initial  states  sj,  •  •  • ,  sJ0  and  success  states  •  •  • ,  .s" .  We 
can  add  a  new  initial  state  so  and  a  new  success  state  ss  with  e-labeled  edges  (so,  s’o").  (1  <  m  <  j )  and 
(ss,.sl)  (1  <  t  <  w). 

Suppose  we  are  also  given  a  finite  set  of  measures  M  =  {mi ,  •  •  • ,  //)/,. }  and  a  function  covers  :  M  — >■  '2A . 
An  atomic  attack  a  E  covers  (mi )  if  adopting  measure  in;  removes  the  atomic  attack  a. 

We  are  now  ready  to  address  the  question  of  what  measures  a  system  administrator  should  deploy  to 
ensure  the  system  is  safe.  Again,  there  is  a  naive  solution,  that  is,  to  try  all  possible  subsets  of  measures 
M '  C  M  and  determine  which  of  those  make  the  system  safe.  We  discuss  this  approach  in  the  context  of 
question  2: 

Question  2:  If  all  measures  for  protecting  a  network  are  deployed,  does  the  system  become  safe? 

Answer:  A  network  administrator  wants  to  find  out  whether  adopting  measures  from  a  set  M '  C  M  will 
make  the  network  safe.  This  question  can  be  answered  in  linear  time  using  the  attack  graph  G.  First,  we  define 
covers(M')  as  (JmeM'  covers  (in).  Next,  we  remove  all  edges  e  from  G  such  that  L(e)  E  covers(M').  The 
network  is  safe  iff  the  success  state  ss  is  not  reachable  from  the  initial  state  so-  This  simple  reachability 
question  can  be  answered  in  time  that  is  linear  in  the  size  of  the  graph. 

As  the  set  of  measures  grows  (and  as  the  set  of  atomic  attacks  grows),  we  really  would  like  to  have  the 
system  administrator  choose  the  smallest  subset  of  measures  that  would  guarantee  the  networked  system  is 
safe.  We  address  this  decision  in  the  context  of  question  3: 

Question  3:  Given  a  set  of  measures  M,  what  is  the  smallest  subset  of  measures  M'  whose  deployment 
makes  the  system  safe? 

Answer:  A  network  administrator  wishes  to  find  a  subset  M'  C  M  of  smallest  size,  such  that  adopting  the 
measures  in  the  set  M '  will  make  the  network  safe.  Unfortunately,  this  problem  is  .Y  /’-complete,  but  we 
develop  good  approximation  algorithms.  We  proceed  in  two  steps: 

Step  1:  Finding  a  small  set  of  atomic  attacks. 

In  this  step,  we  find  a  set  of  atomic  attacks  whose  removal  makes  the  network  safe.  As  described  in  the 
previous  section,  checking  every  possible  subset  of  attacks  is  exponential  in  the  number  of  attacks.  In 
an  earlier  conference  paper  [SHI+02],  we  show  that  finding  the  minimum  set  of  atomic  attacks  which 
must  be  removed  to  thwart  an  intruder  is  in  fact  (VP-complete.  We  repeat  part  of  the  proof  below  (see 
Lemma  2).  We  also  demonstrated  how  a  minimal  set  can  be  found  in  polynomial-time.2  In  this  paper, 
we  further  explore  the  complexity  of  this  problem.  Section  5.2.1  proves  that  the  problem  of  finding 
a  minimum  set  of  attacks  is  polynomially  equivalent  to  the  minimum  hitting  set  problem,  where  the 
collection  of  sets  is  represented  as  labeled  directed  graph.  This  reduction  provided  us  with  additional 
insight.  This  additional  insight  enabled  us  to  find  a  greedy  algorithm  with  provable  bounds. 

2In  the  conference  paper  we  showed  the  reduction  to  the  minimum  cover  problem  [GJ79.  Page  222];  here  we  show  it  to  the  minimum 
hitting  set  problem. 


is  a  function,  where  covers  (mi)  represents  the  set  of  atomic  attacks  that  are  removed  by  adopting  the 
measure  m; .  With  each  attack  a  in  the  set  A' ,  we  associate  a  set  of  measures  M  (a)  which  is  {mi  \  a  G 
cotiers(»77.j)}.  The  set  of  attacks  A'  defines  a  collection  Ca*  of  subsets  of  M .  We  wish  to  find  the 
smallest  subset  M'  C  M  such  that  for  all  a  G  A'  there  exists  an  m8-  G  M'  such  that  a  G  covers (m-i), 
or  equivalently  M'  fl  M (a)  f  0.  This  is  known  as  the  minimum  hitting  set  problem,  which  is  NP- 
complete,  but  good  approximation  algorithms  exist  to  solve  this  problem  (see  Section  5.2.2) 

5.2.1  The  Minimum  Critical  Attack  Sets 

and  the  Minimum  Hitting  Set  Problem 

This  section  addresses  the  first  step  in  the  answer  to  question  3.  Assume  that  we  are  given  an  attack  graph 
G  =  (S,  E,  so ,  ss ,  L),  where  S  is  the  set  of  states,  E  C  S  x  S  is  the  set  of  edges,  so  £  S  is  the  initial  state, 
ss  £  S  is  the  success  state  for  the  intruder,  and  L  :  E  — ►  A  U  {e}  is  a  labeling  function. 

Given  a  state  s  G  S,  a  set  of  attacks  C  is  critical  with  respect  to  s  if  and  only  if  the  intruder  cannot  reach 
his  goal  from  s  when  the  attacks  in  C  are  removed  from  his  arsenal.  Equivalently,  G  is  critical  with  respect 
to  s  if  and  only  if  every  path  from  s  to  the  success  state  ss  has  at  least  one  edge  labeled  with  an  attack  a  G  C. 

A  critical  set  corresponding  to  a  state  s  is  minimum  (denoted  M  (s))  if  there  is  no  critical  set  M'(s)  such 
that  |M'(s)|  <  |M(s)|.  In  general,  there  can  be  multiple  minimum  sets  corresponding  to  a  state  s.  Of  course, 
all  minimum  critical  sets  must  be  of  the  same  size. 

A  critical  set  of  an  attack  graph  G  =  (S,  E ,  so,  ss,  L)  is  defined  as  a  critical  set  corresponding  to  the 
initial  state  so-  Therefore,  the  Minimum  Critical  Set  of  Attacks  (MCSA)  problem  is  the  problem  of  finding  a 
minimum  critical  set  of  attacks  M (so)-  The  decision  version  of  the  problem  is  defined  as  follows:  given  an 
attack  graph  G  =  (S,  E ,  so,ss,L)  and  a  positive  integer  I\,  is  there  a  critical  set  of  attacks  A  C  A  such  that 
|A|  <  /\? 

Lemma  2  Assume  that  we  are  given  an  attack  graph  G  =  (S,  E,  so ,  L)  and  an  integer  k.  The  MCSA  problem 
of  determining  whether  there  is  a  critical  set  C(so)  such  that  |C(so)|  <  k  is  AP-complete. 

Proof:  First,  we  prove  that  the  problem  is  in  NP.  Guess  a  set  C  C  A  with  size  <  k.  We  need  to  check  that 
C  is  a  critical  set  of  attacks.  This  can  be  accomplished  in  polynomial  time  using  the  reachability  algorithm 
described  before  ( see  answer  to  question  2 ).  Therefore,  the  problem  is  in  NP. 

Next,  we  prove  that  the  problem  is  NP -hard.  The  reduction  is  from  the  minimum  hitting  set  problem, 
details  as  given  in  the  remainder  of  this  section. 

Assume  that  we  are  given  an  attack  graph  G  =  (S,  E,  so,  ss,  L).  A  path  it  is  sequence  of  states  q  i,  •  •  • ,  q„ , 
such  that  </;  G  S  and  (</,,  qi+i)  G  E.  A  complete  path  starts  from  the  initial  state  so  and  ends  in  the  success 
state  ss.  The  label  of  a  path  n  =  qi,  ■  ■  ■ ,  qn  (abusing  notation,  we  will  denote  it  also  as  L(tt))  is  a  subset  of  a 
set  of  attacks  A 

n—  1 

U  {L^iAi  + 1)}  \  M  • 

8  =  1 

L(tt)  represents  the  set  of  atomic  attacks  used  on  the  path  n.  A  set  of  attacks  A  C  A  is  called  realizable  in 
the  attack  graph  G  iff  there  exists  a  complete  path  7r  in  G  such  that  L(tt)  =  A.  In  other  words,  an  intruder 
can  use  the  set  of  attacks  A  to  start  from  the  initial  state  and  reach  the  success  state.  The  set  of  all  realizable 
sets  in  an  attack  graph  G  is  denoted  by  Rel(G).  The  following  lemma  is  easy  to  prove  and  follows  straight 
from  the  definitions. 

Lemma  3  Assume  that  we  are  give  an  attack  graph  G  =  (S,  E ,  so,  ss ,  L).  A  set  of  attacks  A  is  critical  iff 

VA'  G  Rel(G).A'  n  A  f  0  . 


In  other  words,  all  realizable  sets  have  a  non-empty  intersection  with  a  critical  set  A. 


in  m 


Lemma  3  proves  that  the  problem  of  finding  whether  the  attack  graph  G  has  a  critical  set  of  size  <  K  is 
the  hitting  set  problem  with  C  =  Rel(G),  S  =  A,  and  K. 

Next  suppose  we  have  an  instance  (( S,  K)  of  the  hitting  set  problem.  We  will  construct  an  attack  graph 
G"  =  (S',  E' ,  Sq ,  s' ,  L'),  where  /.'  :  E'  — »  S  U  {e} ,  i.e.,  the  set  of  attacks  used  in  the  attack  graph  G'  is 
S.  Moreover,  the  set  of  realizable  sets  Rel(G')  of  the  graph  G"  is  the  collection  C.  A  critical  set  of  size 
<  K  of  the  attack  graph  G"  is  a  hitting  set  for  the  collection  G.  Next,  we  describe  the  construction  of  G". 
Let  C  =  {C i,  -  -  - ,  Cm}  be  the  collection  of  sets  and  S  =  { ,s i ,  •  •  - ,  s,,}  be  the  set.  We  make  m  copies 
S'1 ,  •  •  • ,  Sm  of  the  set  S.  The  set  of  elements  in  S1  will  be  denoted  by  {s^ ,  •  •  • ,  s*,  }.  The  set  of  states  S'  in 
the  attack  graph  G"  is 

{s'0,  s' }  U  S1  U---US""  . 

The  initial  state  is  s'0  and  the  final  state  is  s' .  The  set  of  edges  E'  and  the  labeling  function  /.'  are  defined  as 
follows: 

•  There  is  an  edge  from  s'0  to  every  state  in  the  set  {sj ,  s'f,  ■  ■  ■ ,  s'"  } ,  and  label  of  the  edge  (s'0 ,  sj )  is  si 
if  si  G  C{,  otherwise  it  is  e. 

•  For  all  1  <  i  <  m  and  1  <  j  <  n  —  1,  there  is  an  edge  (s®- ,  s®-+1),  and  the  label  of  edge  (s®- ,  s®-+1)  is 
Sj+i  if  Sj+i  Gs'Cj,  otherwise  it  is  e. 

•  There  is  an  edge  from  every  state  in  the  set  {s},  .s{ .  •  •  • ,  s™}  to  the  state  s' ,  and  labels  of  all  these 
edges  is  e. 

The  sizes  of  the  sets  S'  and  E'  in  the  attack  graph  G"  are  mn  +  2  and  2m  +  mn  respectively.  It  is  easy  to 
see  that  Rel(G')  is  equal  to  C,  and  S'  C  S  is  a  critical  set  of  the  attack  graph  G"  iff  S'  is  a  hitting  set  for 
the  collection  C.  Since  the  size  of  G"  is  polynomial  in  the  size  of  the  instance  of  the  hitting  set  problem  and 
the  hitting  set  problem  is  .Y  /’-complete,  the  MCSA  problem  is  .Y  /’-hal'd.  Lemma  2  proves  that  MCSA  is  in 
NP.  Therefore,  MCSA  is  .Y  /’-complete.  The  next  example  illustrates  our  construction. 

Note:  The  discussion  above  also  proves  that  the  problem  of  finding  a  minimum  set  of  measures  whose 
adoption  will  make  the  network  safe  is  also  .Y  /’-complete.  One  can  simply  take  the  set  of  measures  M  to  be 
the  set  of  attacks  A. 

Example  1  We  give  a  short  example  to  illustrate  the  reduction.  Consider  a  set  S'  =  {si,  sn,  S3}-  Suppose 
that  the  collection  G  consists  of  the  following  subsets: 

Gl  =  {si,S2} 

C'2  =  {53,53} 

C3  =  {so} 

The  attack  graph  G"  corresponding  to  this  problem  is  shown  in  Figure  6.  The  set  of  attacks  is  {si,  sn,  S3}. 
The  set  of  realizable  sets  Rel(G')  is  exactly  the  collection  C.  The  set  of  attacks  {si,  so}  is  critical  because 
every  path  from  s'0  to  the  success  state  s's  uses  at  least  one  edge  with  the  label  in  the  set  {si,  si}.  Moreover, 
{si ,  so}  is  a  hitting  set  for  the  collection  C  =  {C 1,  CY,  G3}. 

The  above  discussion  proves  that  the  problem  of  finding  critical  sets  in  attack  graph  is  polynomially 
equivalent  to  finding  hitting  sets  for  a  collection,  with  one  caveat-the  collection  of  sets  C  is  represented  as  an 
attack  graph.  An  attack  graph  can  be  an  exponentially  succinct  representation  of  a  collection  of  sets.  Figure  7 
shows  an  attack  graph  of  linear  size  whose  set  of  realizable  sets  is  the  power  set  of  {si ,  •  •  • ,  sn  } .  Therefore, 
the  minimum  critical  set  problem  is  polynomially  equivalent  to  the  hitting  set  problem  where  the  collection 
of  sets  C  is  represented  as  a  labeled  directed  graph. 


Figure  7:  Attack  graph  representing  an  exponential  number  of  realizable  sets. 


(( S,  K)  be  an  instance  of  the  hitting  set  problem.  Let  S'  and  C  be  initially  the  empty  set.  The  greedy 
algorithm  executes  the  following  step  until  C  =  C . 

•  Pick  an  element  s  out  of  the  set  S\S'  that  covers  the  maximum  number  of  sets  in  the  collection  C  \  C . 

An  element  s  is  said  to  cover  a  set  Si  C  S  iff  s  E  Si . 

•  Let  s  be  the  element  picked  in  the  previous  step  and  (  '  ( .s )  be  the  collection  of  sets  in  C  covered  by  s. 

Update  S'  and  C  as  follows: 

S'  <-  S'  U  {s} 
a  <-  a  uc(s) 

Let  Hci  be  the  d- th  harmonic  number  ^1=  1  J-  Let  C(s)  be  the  number  of  sets  in  the  collection  C  that  are 
covered  by  the  element  s. 

Lemma  4  GREEDY-HITTING-SET is  a  polynomial-time  p ( n ) -approximation  algorithm,  where  p(n)  =  H  (max, e s  { | G ( s )  | } ) . 

The  proof  of  the  lemma  follows  from  the  equivalence  between  the  minimum  hitting  set  and  the  minimum 
cover  problem  [ADP80]  and  the  proof  of  the  approximation  factor  p(n)  for  the  greedy  algorithm  for  the 
minimum  cover  problem  [CLR85].  Using  the  equivalence  between  the  problems  of  finding  a  minimum 
critical  set  and  a  minimum  hitting  set,  we  can  construct  a  greedy  procedure  (called  GREEDY-CRITICAL-SET) 
for  finding  a  critical  set  for  the  attack  graph.  Assume  that  we  are  given  an  attack  graph  G  =  (S,  E,  s o,  ss ,  L), 
where  S  is  the  set  of  states,  E  C  S  x  S  is  the  set  of  edges,  so  £  S  is  the  initial  state,  ss  £  S'  is  the  success 
state  for  the  intruder,  and  L  :  E  — ►  A  U  {e}  is  a  labeling  function.  Moreover,  assume  that  we  can  compute  in 
polynomial  time  the  function  pc  :  ^4  — )-  H,  where  pr;(/i )  is  the  number  of  realizable  sets  in  the  attack  graph 
G  that  contain  the  attack  a.  Formally,  p c (pi)  is  equal  to 

\{A'\a  E  A'  and  A'  E  Rel(G)} \  . 

Initially,  let  A'  be  the  empty  set  and  G'  =  G.  The  greedy  algorithm  GREEDY-CRITICAL-SET  executes  the 
following  step  until  G'  is  empty. 

•  Pick  and  element  a  from  the  set  A  \  A'  that  maximizes  pep  (a). 

•  Let  a  be  the  element  picked  in  the  previous  step.  Update  A'  and  G'  as  follows: 

A!  E-  A!  U  {a} 

Remove  all  edges  labeled  with  a  from  G' 

Lemma  5  GREEDY-CRITICAL-SET  is  a  polynomial-time  p(n  (-approximation  algorithm,  where  p(n)  = 

H  (maxag^t{pG'(a)})- 

Next,  we  explore  conditions  when  the  function  pc  can  be  computed  in  polynomial  time.  Assume  that 
the  attack  graph  G  is  a  DAG.  An  argument  for  this  was  given  in  Section  4.3.  Moreover,  assume  that  each 
atomic  attack  is  used  only  once  on  a  path  from  the  initial  state  so  to  the  success  state  ss.  This  is  not  a 
unreasonable  assumption  because  the  attack  graph  edges  are  labeled  with  instantiations  of  attack  templates 
shown  in  Section  4.3,  e.g.,  a  local-setuid-buffer-overflow  attacks  on  two  different  hosts  are  distinct  in  the 
attack  graph.  Such  attack  graphs  are  called  use-once  DAGs.  The  following  lemma  is  easy  to  prove. 


Lemma  6  For  an  attack  graph  that  is  a  use-once  DAG,  the  function  pc  can  be  computed  in  time  that  is  linear 
in  size  of  the  attack  graph. 


of  thwarting  an  attack?  If  we  have  probabilities  available  to  us,  we  can  annotate  attack  graphs  to  help  system 
administrators  answer  such  questions. 

In  our  work,  we  do  not  require  that  all  transitions  be  given  probabilities;  in  general,  our  annotated  attack 
graphs  can  have  a  mix  of  probabilistic  and  nondeterministic  state  transitions.  We  pursue  the  implications  of 
this  general  kind  of  attack  graph  in  this  section. 

In  general,  we  also  do  not  require  probabilities  to  be  numeric;  they  can  be  symbolic,  e.g.,  “high,” 
“medium,”  or  “low,”  and  even  partially  ordered.  In  an  earlier  paper  [JW01],  we  discuss  an  analysis  that 
uses  symbolic  probabilities;  in  this  paper,  however,  we  restrict  ourselves  to  numeric  values. 


6.1  Probabilistic  Attack  Graphs 

Suppose  that  the  graph  has  a  state  s  with  only  two  outgoing  transitions.  In  a  regular  attack  graph,  the  choice 
of  which  transition  to  take  when  the  system  is  in  state  s  is  nondeterministic.  However,  we  may  have  some 
empirical  data  that  enables  us  to  estimate  that  whenever  the  system  is  in  state  s,  on  average  it  will  take  one  of 
the  transitions  four  times  out  of  ten  and  the  other  transition  six  remaining  times.  We  can  place  probabilities 
0.4and  0.6  on  the  corresponding  edges  in  the  attack  graph.  Intuitively,  the  probability  of  the  transition  s  — >■  s' 
represents  the  likelihood  that  the  atomic  attack  corresponding  to  the  transition  will  succeed.  We  call  a  state 
with  known  probabilities  for  outgoing  transitions  probabilistic.  When  we  have  assigned  all  known  proba¬ 
bilities  in  this  way,  we  are  left  with  an  attack  graph  that  has  some  probabilistic  and  some  nondeterministic 
states  in  it.  We  call  such  mixed  attack  graphs  probabilistic  attack  graphs.  We  use  probabilistic  attack  graphs 
to  evaluate  the  reliability  of  a  network.  Note  that  probabilities  of  all  the  transitions  might  not  be  available 
because  of  lack  of  data,  e.g.,  a  new  type  of  atomic  attack. 

Since  the  attack  graph  includes  only  those  states  and  transitions  that  can  lead  to  success  states,  it  excludes 
some  transitions  that  exist  in  the  complete  model  M .  These  excluded  transitions  can  have  non-zero  proba¬ 
bility,  so  that  the  sum  of  probabilities  of  transitions  from  a  probabilistic  state  will  be  less  than  1.  To  address 
this  problem,  we  must  model  the  rest  of  M  in  some  way.  We  add  a  “catch-all”  escape  state  se  to  the  attack 
graph.  A  probabilistic  state  s  in  the  attack  graph  will  have  a  transition  to  se  if  and  only  if  in  M  there  is  a 
transition  from  s  to  some  state  not  in  the  attack  graph.  The  probability  of  going  from  s  to  se  will  be  1  minus 
the  sum  of  the  probabilities  of  going  to  other  states.  There  are  no  transitions  out  of  se  except  a  self-loop 
(which  preserves  the  totality  of  the  transition  relation  r). 

In  an  attack  graph  containing  the  escape  state  se  attacks  are  allowed  to  terminate  in  se .  We  will  call  them 
escape  attacks,  or  attacks  that  were  pre-empted  by  the  intruder  before  he  reached  his  goal. 

6.1.1  Definition  of  PAGs 

Definition  3  A  probabilistic  attack  graph  or  PAG  is  a  tuple  G  =  (Sn ,  Sq,  se,  S,  t,  zr.  So,  Ss,  L),  where  Sn  is 
a  set  of  nondeterministic  states,  Sq  is  a  set  of  probabilistic  states,  se  £  S  „  is  a  nondeterministic  escape  state 
Oe  ^  Ss),  S  =  Sn  U  Sq  is  the  set  of  all  states,  t  C  S  x  S  is  a  transition  relation,  7r  :  Sq  — >■  S  — >■  3?  are 
transition  probabilities.  So  C  S  is  a  set  of  initial  states,  Ss  C  S'  is  a  set  of  success  states,  and  L  :  S  — >■  2AP 
is  a  labeling  of  states  with  a  set  of  propositions  true  in  that  state. 

A  probabilistic  attack  graph  distinguishes  between  nondeterministic  states  (set  Sn )  and  probabilistic  states 
(set  Sq).  Moreover,  the  sets  of  nondeterministic  and  probabilistic  states  are  disjoint  (S  n  fl  Sq  =  0).  The 
function  tt  specifies  probabilities  of  transitions  from  probabilistic  states,  so  that  for  all  transitions  si  — >■ 
so  E  t  such  that  si  E  Sq,  we  have  P(s i  — ►  sn)  =  tt(si)(s2)  >  0.  Thus,  7r(s)  can  be  viewed  as  a 
probability  distribution  on  next  states.  Intuitively,  when  the  system  is  in  a  nondeterministic  state  sn,  we 
have  no  information  about  the  relative  probabilities  of  the  possible  next  transitions.  When  the  system  is  in  a 
probabilistic  state  sq,  it  will  choose  the  next  state  according  to  probability  distribution  n  ( sq ) . 


Let  G  =  (S,  t ,  So,  Ss ,  L)  be  the  attack  graph  and  P  a  function  that  assigns  probabilities  to  transitions.  The 
probabilities  can  be  loosely  interpreted  as  the  probability  of  the  atomic  attack  corresponding  to  the  transition 
succeeding.  We  are  interested  in  finding  the  reliability  of  the  attack  graph,  i.e.,  the  probability  that  the 
intruder  will  not  succeed.  We  can  view  G  as  a  Markov  chain  with  S  as  its  state  space  and  /’(.s  — >■  s2)  as 
its  transition  probability.  Let  U  :  S  — >  )i+  be  the  steady  state  probability  of  the  Markov  chain  (see  [Dur95] 
for  definitions  and  technical  conditions).  In  this  case,  the  reliability  of  the  attack  graph  G  is  given  by  the 
following  expression: 

s£Ss 

In  other  words,  the  reliability  is  the  probability  that  in  the  “long  run”  the  Markov  chain  will  not  be  in  a  state 
in  the  set  Ss . 

In  general,  however,  we  do  not  have  probabilities  assigned  to  all  transitions;  thus  in  Section  6.2  we  show 
how  to  perform  similar  reliability  analysis  on  probabilistic  attack  graphs  in  the  presence  of  nondeterministic 
states.  The  justification  of  our  approach  relies  on  converting  a  probabilistic  attack  graph  (PAG)  into  an 
alternating  probabilistic  attack  graph  (APAG)  and  then  interpreting  the  result  as  a  Markov  Decision  Process; 
we  give  this  construction  and  interpretation  in  Section  6.3;  we  give  the  proof  of  correctness  of  the  MDP  value 
iteration  algorithm  applied  to  PAGs  in  Section  6.4.  Sections  6.3  and  6.4  can  be  skipped  upon  a  first  reading. 


6.2  Reliability  Analysis  of  PAGs 

Assume  that  we  are  given  a  PAG  G  =  (Sn ,  Sq,  se ,  S,  t,  it,  So,  Ss  ,  L).  Intuitively,  we  are  interested  in  finding 
out  the  probability  that  the  intruder  will  reach  a  success  state  starting  from  one  of  the  initial  states.  As  shown 
above,  in  the  absence  of  nondeterministic  states  we  can  compute  this  metric  by  using  the  steady  state  prob¬ 
abilities  of  the  Markov  chain.  In  the  presence  of  nondeterministic  states  the  intruder  will  choose  transitions 
in  order  to  maximize  his  probability  of  succeeding.  For  example,  if  an  intruder  reaches  a  nondeterministic 
state  s  with  transitions  to  si ,  •  •  • ,  s^.,  he  will  choose  to  transition  to  state  s8-  (1  <  i  <  n)  which  will  maximize 
his  probability  of  reaching  a  success  state.  This  idea  can  be  “formalized”  using  concepts  from  the  theory  of 
Markov  Decision  Processes  [Alt99,  Put94], 

6.2.1  Value  Iteration  for  PAGs 

Given  a  state  s,  the  set  of  successors  of  s  is  denoted  by  siicc(s).  Formally,  siicc(s)  is  equal  to  {s'|  (s,  s' )  £  r}. 
First,  we  define  a  value  function  V  :  S  — >■  3r,+  .  For  alls  £  Ss,  V'(s)  =  1.0.  For  all  states  s  £  S\Ss  the  value 
function  is  iterated  according  to  the  following  equations  until  convergence. 

t r,  *  _  f  1 1 1 h xs / s i  I  (s  )  if  s  G  Sn  \  Ss 

HS)  ~  l  E s,esucc{s)P(s^s>)V(s>)  if s£Sq\S, 

Let  I  ’ '  be  the  value  function  after  convergence.  Intuitively,  Ese,s’0  ' ' r  ( -s )  is  the  probability  for  the 
intruder  to  reach  a  success  state  if  he  “breaks”  the  nondeterminism  to  maximize  the  probability  of  succeeding. 
Therefore,  the  worst  case  reliability  of  the  network  is  1  —  s0  i  *(s)-  This  algorithm  is  known  as  value 
iteration.  The  justification  of  the  value  iteration  algorithm  as  applied  to  PAGs  is  presented  in  Section  6.4. 

6.2.2  Example  Revisited 

We  implemented  the  value  iteration  algorithm  in  our  attack  graph  post-processor  and  ran  it  on  a  slightly 
modified  version  of  the  intrusion  detection  example  from  Section  4.  In  the  modified  example,  each  attack 
has  both  detectable  and  stealthy  variants.  The  intruder  chooses  which  atomic  attack  to  try  next,  and  he  has 
a  certain  probability  of  picking  a  stealthy  or  a  detectable  variant.  We  assigned  imaginary  probabilities  of 
picking  a  stealthy  attack  variant  as  follows:  0.2  for  sshd  buffer  overflow,  0.5  for  ftp  .rhosts,  0.05  for  the 


In  this  setup,  the  computed  probability  of  intruder  success  is  0.2,  and  his  best  strategy  is  to  attempt  sshd 
buffer  overflow  on  host  ip i,  and  then  conduct  the  rest  of  the  attack  from  that  host.  The  only  possibility  of 
detection  is  the  sshd  buffer  overflow  attack  itself,  since  the  IDS  does  not  see  the  activity  between  hosts  ip i 

and  ip  2  . 

Given  this  context,  a  system  administrator  can  answer  the  following  question: 

Question  4:  The  deployment  of  which  security  measure(s)  will  increase  the  likelihood  of  thwarting  an  at¬ 
tacker? 

Answer:  Installing  an  additional  IDS  component  to  monitor  the  network  traffic  between  hosts  i\>  and  ipo 
reduces  the  probability  of  the  intruder  remaining  undetected  to  0.025;  installing  a  host-based  IDS  on  host  ipo 
reduces  the  probability  to  0.16.  Other  things  being  equal,  this  is  an  indication  that  the  former  remedy  is  more 
effective. 


6.3  Alternating  Probabilistic  Attack  Graphs  and  Markov  Decision  Processes 

In  this  section  we  show  that  probabilistic  attack  graphs  can  be  reduced  to  Markov  Decision  Processes  (without 
the  reward  function).  We  then  demonstrate  how  we  can  assign  a  reward  function  to  attack  graphs  such  that 
standard  MDP  algorithms  can  be  used  to  compute  reliability  metric  of  the  network  being  modeled. 

Definition  4  [Alt99,  Put94]  A  Markov  Decision  Process  is  a  tuple  (X,  A,  V ,  c)  where 

•  X  is  a  finite  state  space.  Generic  notation  for  MDP  states  will  be  *,  y, 

•  A  is  a  finite  set  of  actions.  A(*)  C  A  denotes  the  actions  that  are  available  at  state  x.  Set  K  = 
(a-,  a)  :i£X,oE  A(*)  is  the  set  of  state-action  pairs.  A  generic  notation  for  an  action  will  be  a. 

•  V  :  X  x  A  x  X  are  the  transition  probabilities;  thus,  V(xay)  (also  written  as  V,ray)  is  the  probability 
of  moving  from  state  x  to  y  if  action  a  is  chosen. 

•  r  :  K  — >  3?  is  an  immediate  reward.  Cost  may  be  equivalently  viewed  as  a  negative  reward.  We  will 
freely  use  the  term  cost  to  mean  negative  reward,  and  vice  versa. 

An  execution  fragment  (also  known  as  history  in  the  traditional  MDP  literature)  of  an  MDP  is  a  sequence 
A’oaii’i  •  •  •  anxn  of  alternating  states  and  actions  such  that  the  sequence  begins  and  ends  with  a  state,  and 
for  all  0  <  k  <  n,  au  E  A(*fc_i)  and  0  <  T(xk-i,au,Xk)  <  1.  Given  an  execution  fragment  e  = 
*oOi*i . .  .a„xn,  the  probability  of  the  execution  fragment  (denoted  by  P(e))  is  given  by  the  following 
expression: 

n 

C  (*/,._  1  ,  O/,.  ,  */,;  ) 

k  =  1 

It  is  possible  to  convert  a  probabilistic  attack  graph  into  an  MDP  such  that  the  behaviors  of  the  PAG  and 
the  MDP  are  identical.  To  explain  the  conversion  procedure,  we  define  a  restricted  kind  of  probabilistic  attack 
graph. 

Definition  5  An  alternating  probabilistic  attack  graph  or  APAG  is  a  tuple  G  =  (Sn  ,Sq,se,S,  t„  ,  Tq ,  tt,  So  , 
Ss,L),  where  Sn  is  a  set  of  nondeterministic  states,  Sq  is  a  set  of  probabilistic  states,  se  E  Sn  is  a  nonde- 
terministic  escape  state,  S  =  Sn  U  Sq  is  the  set  of  all  states,  rn  C  S  „  x  Sq  is  a  set  of  nondeterministic 
transitions,  rq  C  Sq  x  Sn  is  a  set  of  probabilistic  transitions,  7r  :  Sq  — >■  Sn  -P  h'  are  transition  probabilities. 
So  C  S'  is  a  set  of  initial  states.  S',  C  S  is  a  set  of  success  states,  and  L  :  S  — >■  2AP  is  a  labeling  of  states 
with  a  set  of  propositions  true  in  that  state. 
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Figure  8:  Converting  PAG  to  APAG 


An  alternating  probabilistic  attack  graph  (APAG)  does  not  have  any  transitions  between  two  nondeterministic 
or  between  two  probabilistic  states.  In  other  words,  a  nondeterministic  state  has  transitions  to  probabilistic 
states  only,  and  vice  versa.  An  execution  of  an  APAG  will  always  have  strictly  alternating  nondeterministic 
and  probabilistic  states. 

Next  we  describe  an  algorithm  that  converts  a  PAG  GP  =  (Sn,  Sq,  se,  S,  t,  7r,  S  o,  Ss,  L)  into  an  APAG 
Gp  =  ( SA  ,  SA ,  se ,  S,  ta .  r/;' .  7t  ' .  ,S'!:i .  Ss ,  LA )  that  has  equivalent  behaviors.  The  algorithm  works  by  adding 
hidden  states  and  transitions  to  the  graph  such  that  every  execution  becomes  strictly  alternating,  yet  does  not 
change  its  obsen’able  (non-hidden)  components. 

We  start  with  SA  =  Sn,  SA  =  Sq,  ta  :=  0,  ta  :=  0,  tta  :=  0.0,  and  LA  =  L.  Next, 

1.  Whenever  t  has  a  transition  from  probabilistic  state  Si  to  nondeterministic  state  S2>  we  add  the  transi¬ 
tion  to  ta  and  its  probability  to  nA . 

2.  Whenever  t  has  a  transition  from  nondeterministic  state  si  to  probabilistic  state  so,  we  add  the  transi¬ 
tion  to  ta  . 

3.  Whenever  r  has  a  transition  between  two  nondeterministic  states  si  and  so,  we  add  a  hidden  proba¬ 
bilistic  state  Sh  to  SA,  an  observable  transition  si  — ►  to  ta,  and  a  hidden  transition  $h  — >■  so  to  ta, 

assigning  the  latter  probability  1.0  in  irA  (Figure  8a).  We  also  set  LA(su )  =  L(s i ). 

4.  Whenever  r  has  a  transition  between  two  probabilistic  states  si  and  so,  we  add  a  hidden  nondeter¬ 
ministic  state  Sh  to  SA,  a  hidden  transition  */,  — >■  so  to  ta,  and  an  observable  transition  si  — ►  s/,  to 
ta,  assigning  the  latter  the  original  probability  p  of  going  from  si  to  so  (Figure  8(b)).  We  also  set 
LA(.sh)  =  L(.s1). 


Let  GP  be  a  PAG  and  GA  be  the  corresponding  APAG.  An  execution  fragment  e  =  sosi  •  •  •  sn  in  GA  is  called 
proper  if  the  start  and  end  states  (so  and  sn)  are  observable  states.  Let  e  be  a  proper  execution  fragment  of 
GA.  We  define  eobs  by  removing  hidden  states  and  hidden  transitions  from  e,  i.e.,  restricting  the  execution 
to  observable  states  and  transition.  Consider  an  execution  fragment  e  =  sosi  •  •  •  sn .  Let  Sp  (e )  be  the  set  of 
probabilistic  states  in  the  set  {.s,:, .  •  •  • ,  s„_i}.  Define  the  probability  of  an  execution  fragment  e  (denoted  by 
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Figure  9:  Converting  an  APAG  to  a  MDP 


P(e))  as 


n  ~ ^  s*+i )  • 

« i  £Sp(e) 


In  other  words,  the  probability  of  an  execution  fragment  is  the  product  the  probabilities  of  the  probabilistic 
transitions  in  it.  The  following  lemma  follows  straight  from  the  construction. 


Lemma  7  Let  GP  be  a  PAG  and  be  the  corresponding  APAG.  Let  e  be  a  proper  execution  fragment  of 
Gp  .  The  following  three  statements  are  true: 

1.  eobs  is  an  execution  fragment  of  Gp. 

2.  P(e)  =  P(eobs),  where  the  first  probability  is  interpreted  in  Gp  and  the  second  probability  is  inter¬ 
preted  in  GP . 

3.  For  all  execution  fragments  e  i  of  GP  there  exists  proper  execution  fragment  e  in  Gp  such  that  e  =  <  'f'x . 


Lemma  7  clearly  shows  that  there  is  a  one-to-one  correspondence  (given  by  ob-s )  between  proper  execution 
fragments  of  a  APAG  and  corresponding  execution  fragments  of  a  PAG.  Moreover,  this  correspondence 
preserves  probabilities.  We  have  shown  that  APAGs  have  the  same  expressive  power  as  PAGs,  so  hereafter 
we  consider  them  interchangeable. 

An  APAG  G  =  (Sn ,  Sq,  se,  S,  rn ,  rq,  n,  So,  Ss ,  L),  has  a  direct  interpretation  as  an  MDP  Mq  =  (X,  A,  V ,  c), 

where  X  =  Sn ,  A  =  t„,  .  That  is,  each  action  in  the  MDP  represents  a  transition  from  a  nondeterministic  to 
a  probabilistic  state.  Further,  let  x,  y  £  X  and  a  £  .  I  (.r),  so  that  a  represents  a  transition  from  x  to  some 
probabilistic  state  sq  in  the  APAG.  Then  we  have  V(x,  a,  y)  =  n (sq)(y). 

It  is  preferable  to  have  all  APAG  success  states  represented  explicitly  as  MDP  states,  so  that  we  can  reason 
about  attacks  in  the  MDP  context.  For  this  reason,  we  add  a  hidden  nondeterministic  state  (and  a  transition 
thereto)  to  every  probabilistic  success  state  in  the  APAG.  We  omit  proofs  of  equivalence  of  an  APAG  before 
and  after  this  modification. 

Figure  9(a)  shows  an  example  APAG,  with  the  corresponding  MDP  shown  in  Figure  9(b).  The  nonde¬ 
terministic  transitions  from  the  root  node  in  the  APAG  are  represented  by  the  MDP  actions  a,  b,  and  c.  The 
leftmost  leaf  in  the  APAG  is  a  probabilistic  success  state;  in  the  MDP  it  is  represented  by  the  appended  hidden 
nondeterministic  state. 


the  reward  function  r  depending  on  the  questions  we  are  trying  to  answer. 

Let  e  =  sjj  s’js"  •  •  •  s"  ~ 1  s(’  s”  be  an  execution  fragment  of  the  APAG  G,  where  s£  and  s|  represent 
nondeterministic  and  probabilistic  states  respectively.  Let  mdp(e)  =  emdp  =  •  •  •  sS-  where  tf  is  the 

action  that  corresponds  to  the  transition  sf_  1  — >■  s1- .  Notice  that  in  mdp(e)  probabilistic  states  do  not  occur. 
The  proof  of  the  following  lemma  follows  straight  from  the  construction. 

Lemma  8  Let  G  be  a  APAG  and  Mq  be  the  corresponding  MDP  Let  e  be  an  execution  fragment  of  G  and 
mdp(e)  be  the  corresponding  execution  fragment  in  the  MDP  Mr; .  The  following  statements  are  true. 

1.  mdp(e)  is  an  execution  fragment  of  the  MDP  Mg- 

2.  P(e)  =  P (mdp(e)) ,  where  P(e)  and  P(mdp(e))  are  interpreted  in  G  and  Mq  respectively. 

3.  For  all  execution  fragments  em  in  the  MDP  Mg,  there  exists  an  execution  fragment  e  in  G  such  that 

mdp(e)  =  em. 


6.4  Correctness  of  the  Value  Iteration  Algorithm  for  Attack  Graphs 


Let  G  =  (Sn  ,Sq,-se,S,  t ,  7 r,  So,  Ss,L)  be  a  PAG,  and  GA  =  (SA ,  SA 


,Sa,ta,tatta,S0,Ss,La)  be 


the  corresponding  APAG.  Recall  that  the  APAG  GA  is  obtained  from  the  PAG  G  by  adding  hidden  states 
whenever  there  is  a  transition  between  two  nondeterministic  or  probabilistic  states  (see  Section  6.3).  An 
APAG  G  =  ( Sn ,  Sq ,  se ,  S,  t„,  ,Tp,ir,  So,  Ss,  L)  has  a  direct  interpretation  as  an  MDP  Mg  =  ( X ,  A ,  V ,  r) , 
where  X  =  Sn,  A  =  rn.  That  is,  each  action  in  the  MDP  represents  a  transition  from  a  nondeterministic  to 
a  probabilistic  state.  Further,  let  x,  y  £  X  and  a  £  .  I  (.r),  so  that  a  represents  a  transition  from  x  to  some 
probabilistic  state  sq  in  the  APAG.  Then  we  have  V(x,  a,  y)  =  7r(sq)(y).  We  first  demonstrate  that  the  value 
iteration  algorithm  (or  VI  for  short)  on  the  APAG  GA  is  simply  a  transformed  version  of  the  value  iteration 
algorithm  on  the  corresponding  MDP  Mq  with  an  appropriate  reward  function  r.  After  that,  we  prove  that 
the  value  iteration  algorithm  on  the  PAG  and  the  corresponding  APAG  converge  to  the  same  value.  The 
advantage  of  this  approach  is  that  all  the  technical  results  in  the  context  of  value  iteration  in  MDPs  can  be 
directly  applied  to  value  iteration  in  PAGs  [Put94,  Chapter  9], 


6.4.1  Correspondence  Between  Value  Iteration  in  MDPs  and  APAGs 

Consider  a  MDP  M  =  (X,  A,  V ,  r).  A  value  function  is  positive  real  valued  function  V  :  X  — ►  3r,+  .  The 
value  iteration  algorithm  uses  the  following  equation  to  update  the  function  S  ’: 

V(x)  =  max  [r(*,a)+  )  V(x ,  a,  y)V(y)] 

aeA(.r) 

y£X 

Technical  conditions  that  guarantee  the  convergence  of  the  value  iteration  algorithm  can  be  found  in  [Put94, 
Chapter  9] . 

Let  Ga  be  a  APAG  and  Mq  be  the  corresponding  MDP.  Recall  that  we  assumed  that  all  success  states  in 
Ga  are  nondeterministic  states  so  that  they  are  explicitly  represented  in  the  MDP  Mq-  Before  we  proceed,  we 
need  to  slightly  modify  the  MDP  Mg-  We  add  a  new  state  snew  and  action  anew  to  the  MDP  Mr;.  The  only 
action  allowed  from  snew  is  anew  (A(snet(I)  =  {a„e,„})andP(s,ie,lha„ett„s,le,ll)  =  1.0  (so  by  definition 
P(snew ,  anew ,  s)  =  0.0  if  s  f  snew)-  Moreover,  we  add  the  action  anew  to  the  action  set  corresponding  to  the 
success  states  Sf  and  for  alls  £  Sf  we  have  P(s,  anew ,  snew )  =  1.0  (so  by  definition  P(s,  anew,  s')  =  0.0 
if  s'  f  Snew)-  We  have  the  following  reward  function  r 

.  ,  f  1.0  if  s  £  S',  and  a  =  anew 

r(s’a)  =  1  0.0  otherwise 


state  s„ew  and  1.0  to  a  state  in  the  set  Sf .  For  states  that  are  not  in  the  set  {snet(,}  U  Ss  the  value  function  V 
changes  according  to  the  following  equation: 

V{x)  =  max  V  P(x,a,y)V(y) 

a£A(x)  z ' 
v  }yex 

=  max  V  P(sq  ->■  y)V(y) 

S  q£SUCC{X'  ^ 


y  ex 


The  second  equation  follows  from  the  construction  of  the  MDP  Mg  from  the  APAG  GA .  Recall  that  actions 
in  the  MDP  correspond  to  the  tr  ansitions  from  nondeterministic  to  probabilistic  states.  Next  we  extend  the 
value  function  V  to  probabilistic  states  Sq  by  defining  V(s)  (for  all  s  £  Sq)  as 

^2  p(-s  y)V(y)  • 
y  ex 

Notice  that  in  an  APAG  only  successors  of  a  probabilistic  state  s  are  nondeterministic  state,  so  V(y)  is  well 
defined.  Using  this  definition  the  value  iteration  algorithm  can  be  re-written  as: 


V(8)  = 


_  f  succ(s)  V(s') 


if  8  6  Sn  \  Ss 


E  s-esucc^Pi^s'Wis')  if  sesq\s. 


The  value  iteration  (VI)  equation  given  above  was  obtained  by  transforming  the  VI  equation  for  the  corre¬ 
sponding  MDP.  Moreover,  the  equation  we  obtain  is  exactly  the  VI  equation  for  an  APAG  that  was  provided 
earlier  (see  Section  6.2). 

6.4.2  Correspondence  Between  Value  Iteration  in  MDPs  and  PAGs 


Let  G  =  (S„  ,Sq,se,S,  t,  7 r,  So,  Ss,L )  be  a  PAG,  and  GA  =  (SA ,  SA 


,SA 


TA  TA  7TA 
'  1  n  i  1  q  i  n  ' 


So,Ss,LA)  be 

the  corresponding  APAG.  Recall  that  GA  is  obtained  from  G  by  adding  hidden  states  whenever  there  is  a 
transition  between  two  nondeterministic  or  probabilistic  states  (see  Figure  8).  Suppose  there  is  a  transition 
between  two  nondeterministic  states  si  and  .sL,  in  G.  In  GA,  we  add  a  new  probabilistic  state  .s/,  and  add 
transitions  si  — ►  .s /,  and  .s /,  — >■  s2,  where  the  probability  of  the  transition  .s /,  — >■  s2  is  1.0.  Consider  the  *-th 
iteration  of  the  VI  algorithm  in  G.  In  this  case,  the  value  l  ’  (.s2 )  in  the  (i  —  1  )-the  iteration  is  used  to  update 
the  value  of  the  state  si .  Now  consider  the  value  iteration  algorithm  in  GA .  The  value  I  ’  (.s/, )  of  the  hidden 
state  .s/,  in  the  (i  —  l)-th  iteration  is  used  to  update  the  value  of  V(si )  in  the  *-th  iteration.  It  is  easy  to  see 
that  V(sh )  in  the  (i  —  l)-th  iteration  is  V(s2)  in  the  (i  —  2)-th  iteration.  Therefore,  hidden  states  add  a  delay 
of  1  in  the  value  iteration  algorithm.  The  case  for  transition  between  two  probabilistic  states  is  analogous. 

Consider  a  PAG  G  =  (Sn,  Sq,  se,  S,  r,  tt,  So,  Ss,  L).  The  equation  for  the  value  iteration  algorithm 
without  delay  is: 

l.o  ifseSs 

maxs/ ^ succ( s )  1  (s  )  if  s  G  Sn  \  Ss 


\n 


Es'e 


,P{s  ->  sOV-^s')  if  seSq\S, 


js'  £succ(s) 

We  have  added  the  iteration  index  i  to  the  VI  algorithm  so  that  we  can  refer  to  it  in  the  proof.  The  value 
iteration  algorithm  with  the  delay  is: 


= 


1.0 

m a x{ m -i x.s- > a_ , ,//  /  , , n s n  t  \ 
G  succ(  s  )ns. 


V?  2  ( s' ) ,  max,  -  e  succ  ( s )  n  sq  Vl  1  ( s' ) } 


if-  Ss 
if  s  G  Sn  \  Ss 


P(s  ^  S')VT  V)  +  Es'6s«cc(s)ns„  p(s  sW  V)  if®  e  Sq  \  S. 


Initially,  both  sequences  start  with  the  value  functions  V°  and  V®  that  assign  1.0  to  states  in  Sf  and  0.0  to  all 
other  states.  Notice  that  in  the  value  iteration  algorithm  for  V[  there  is  delay  of  1  added  (the  (i  —  2)-th  value 


■s  £  S  and  i  >  2: 

V'8'(s)  >  V'/(s)  >  Vi_2(s) 

The  equation  given  above  directly  follows  from  the  monotonicity  property  and  the  equations  that  define  value 
iteration. 

Suppose  V  converges  to  l  4  pointwise,  i.e.,  for  all  s  £  S,  V'(s)  — >■  I4(s).  Next  we  prove  that  for  all 
s  £  S,  if  V'*(s)  — ?>  14 (s|y  then  1  1  - 1  — >■  14 (s).  This  proves  that  Vi  also  converges  to  14 .  By  definition  of 
convergence,  for  all  e  >  0,  there  exists  a  positive  integer  N (e)  such  that  for  all  *  >  N (e)  we  have 

|14(s)-V8(s)|  <  e. 

Assume  that  we  are  given  a  ft  >  0.  It  is  easy  to  see  that  the  limit  14(s)  >  V1  (s)  for  all  *  (this  follows  from 
the  fact  that  V%  (s)  is  a  monotonic  sequence).  Therefore,  we  have  the  following  inequality 

|Vi(s)  -  Vl(s)\  <  |14(s)  -  Vi_2(s)|  . 

The  equation  given  above  follows  from  the  inequality  I  )'  ( .s )  >  V'*_2(s)  for  all  s.  Since  V'*(s)  — ►  14(s), 
there  exists  an  N  (f3)  such  that  if  *  >  N  (ft),  then 

|14(s)-V8(s)|  <  ft- 

By  the  argument  given  above  |14(s)  —  l'4(s)|  <  ft  for  i  >  N (/3)  +  2.  This  proves  that  I? («)  -►  14 (s). 
Conversely  assume  that  14  converges  to  \ft.  Using  the  inequality  given  below  it  is  easy  to  prove  that  V1  (s)  — > 

K(*h 

|v;'(s)-u8(s)|  <  |i4'(s)  -  v? («)l 

Therefore,  we  prove  that  the  value  iteration  algorithm  with  and  without  delay  converge  to  the  same  value. 
The  VI  algorithm  with  delay  is  essentially  the  VI  algorithm  on  the  APAG  GA ,  which  was  derived  from  the  VI 
algorithm  on  the  corresponding  MDP.  Therefore,  the  correctness  of  the  VI  algorithm  on  the  PAG  G  follows. 


7  Summary  of  Contributions  and  Future  Work 

Our  foremost  contribution  is  the  automatic  generation  of  attack  graphs.  Our  key  insight  is  that  an  attack  is 
equivalent  to  a  counterexample  produced  by  off-the-shelf  model  checkers;  the  attack/counterexample  is  a 
witness  to  a  violation  of  a  safety  property.  By  a  small,  but  critical  enhancement  to  an  existing  model  checker, 
i.e.,  NuSMV,  we  can  easily  produce  attack  graphs  automatically;  moreover,  these  graphs  are  succinct  and 
exhaustive.  A  by-product  of  this  part  of  our  work  is  showing,  by  example,  what  level  of  abstraction  is 
appropriate  for  modeling  attacks.  We  use  simple  state  machine  specifications  to  model  not  just  intruder 
behavior  (by  a  set  of  atomic  attacks),  but  also  normal  system  behavior,  system  administrator  recovery  actions, 
and  connectivity  (communication)  between  subsystems. 

Our  second  most  important  contribution  is  support  for  a  range  of  formal  analyses  of  attack  graphs.  Se¬ 
curity  analysts  use  attack  graphs  informally  for  attack  detection,  defense,  and  forensics.  In  this  paper,  we 
explain  how  they  can  now  use  our  minimization  analysis  technique  on  attack  graphs  to  more  precisely  an¬ 
swer  questions  like  “Which  security  measure  should  I  deploy  in  order  to  thwart  this  set  of  attacks?”  and 
“Which  set  of  security  measures  should  I  deploy  to  guarantee  the  safety  of  my  system?”  To  do  reliability 
analysis,  we  annotate  attack  graphs  with  probabilities  and  then  interpret  them  as  Markov  Decision  Processes 
(MDP).  Then,  by  using  MDP  algorithms  such  as  value  iteration,  security  analysts  can  more  precisely  answer 
questions  like  “Will  deploying  this  intrusion  detection  system  increase  or  decrease  the  likelihood  of  thwarting 
this  type  of  attack?” 

On  the  theoretical  front,  we  have  so  far  restricted  our  work  to  only  safety  (invariant)  properties.  To  exploit 
the  full  power  of  model  checking,  we  need  a  method  of  generating  attack  graphs  for  more  general  classes 


AG  (server  .user  .request  — >■  AF  (server. user. acesss ) ) 

This  property  would  not  be  true  if  the  server  can  be  disabled  using  a  denial-of-service  attack.  Another  such 
liveness  property  is  that  a  legitimate  user’s  transaction  will  finish  despite  intruder  interference.  We  plan  to 
explore  generation  of  attack  graphs  for  universally  quantified  fragments  of  Computational  Tree  Logic  and 
Linear  Temporal  Logic. 

On  the  practical  front,  we  plan  to  conduct  larger  case  studies  to  illustrate  the  usefulness  of  automatically 
generating  attack  graphs.  To  make  our  tool  suite  more  usable  by  security  experts  and  system  administrators, 
we  see  the  value  of  building  a  library  of  specifications  of  atomic  attacks.  Our  hope  is  that  increasing  this 
arsenal  of  specifications  outpaces  the  growth  in  the  arsenal  of  known  attacks;  we  can  potentially  discover 
new,  unexpected  attacks,  and  hence  identify  new  network  vulnerabilities.  Finally,  we  also  intend  to  build  a 
tool  that  merges  our  work  on  attack  graphs  with  existing  intrusion  detection  technologies.  The  tool  is  intended 
help  security  analysts  evaluate  and  enhance  the  security  of  a  network. 
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